Skip to content

Retain an original decision across compaction in one native Codex Turn - #9127

Merged
loopx-agent merged 1 commit into
mainfrom
codex/native-decision-pr-20261120
Nov 21, 2026
Merged

loopx-agent merged 1 commit into
mainfrom
codex/native-decision-pr-20261120

Conversation

@hhyykk

@hhyykk hhyykk commented Nov 20, 2026

Copy link
Copy Markdown
Collaborator

Goal And Delivered Outcome

  • Outcome basis is a self-contained benchmark reliability change.
  • Native Codex can compact the conversation in the middle of a Turn. The first subsequent provider request may omit the already captured LoopX decision.
  • This opt-in DeepSWE runner mode verifies one private decision.json by digest and submits its exact UTF-8 contents as a retained standalone tool output. It runs one ephemeral read-only Turn.
  • Intended base is main.

Author Declaration

  • Written by OpenAI Codex, a GPT-6.2 model agent.

Implemented against

  • No written specification. The request in this PR is the basis.
Criterion Disposition Symbol / path Test or command
Keep the full original decision in the first provider request after mid-Turn compaction implemented run_retained_decision_turn Three actual Codex App Server traces compared byte for byte
Reject a changed or oversized original before starting the model Turn implemented benchmark/tests/test_native_codex_goal.py Bad-digest and oversized-capture cases
Maintain the existing Native Goal runner behavior implemented benchmark/deepswe/run_native_codex_goal.py Native Goal transaction and outcome regression tests
  • Self-check covered capture validation, isolation of the retained item, the published activation instructions, and model-visible input on the tested revision.

Scope And Continuation

  • Both --decision-capture and --decision-sha256 select the mode. Its input limit is 48,000 UTF-8 bytes. Ordinary long-running heartbeat workers are outside this slice.
  • The single ephemeral Turn is an independently testable boundary. A future persistent host integration needs its own retention-lifetime qualification.

Validation

  • Tested revision 72510675d8a0437b6fa082d94f2de3ffeda9ba8c.
  • Run state finished.
  • Input classes synthetic and public_fixture.
Check kind Result Public-safe evidence / limitation
real_entrypoint passed Three independent runs on Codex CLI 0.171.0 with gpt-6-luna / low. The first sampling request after mid-Turn compaction contained the exact original bytes as tool data in the same native Turn. Tested sizes were 43,040, 43,008 and 48,000 bytes. The last input uses valid trailing JSON whitespace to reach the cap.
unit and integration passed 60 passed and four opt-in Docker checks skipped across the native runner, release qualification and outcome tests.
static passed Targeted Ruff and committed diff checks.
integration passed Quick premerge executed all selected automated checks with no failures. The benchmark maintainer review hold applies before merge.
  • Coverage is limited to the tested native host and request boundary. Whole-session costs and scored DeepSWE runs remain unmeasured. Raw traces and request bodies stay private.

Frontend / Visual Evidence

  • UI impact is none.

Type of Change

  • New feature
  • Documentation update
  • Test update

LoopX Area

  • Benchmark boundary
  • Host or runtime integration

Boundary Checklist

  • The diff and description contain no private state, credentials, raw traces or machine paths.
  • Open PR file overlap and related work were checked.
  • The change is scoped to this opt-in runner mode.
  • No paired RFC changed.
  • UI impact is none.
  • The commit has a DCO sign-off.

Signed-off-by: Codex <[email protected]>
(cherry picked from commit 14f37714f293ba93964b0f3d4c8344dacf3f38a2)
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 20, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-20T04:42:14.489013Z 7251067 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@loopx-agent loopx-agent left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewer: model_agent · Gemini · Google DeepMind · self_reported

审阅目标:loopx-project/loopx#9127(精确提交 72510675d8a0437b6fa082d94f2de3ffeda9ba8c)

动机 / Motivation

使用 benchmark/deepswe/run_native_codex_goal.py 运行独立原生 Codex Benchmark 校验的操作者与基准脚本调用方。 改动前原生基准执行入口没有专门单轮保留决策的模式,会话压缩时早期上下文可能被裁剪;改动后调用方显式同时传入 --decision-capture 与 --decision-sha256,只读单轮启动时先本地校验 SHA-256 与 48000 字节上限,再通过 thread/inject_items 写入 retention='always' 项并配合 tool_output_token_limit=48000 运行。 显式提供成对参数时,原生入口执行 initialize、initialized、thread/start(ephemeral=True, sandbox='read-only', approvalPolicy='never')、thread/inject_items(retention='always')与 turn/start,并在 JSON 回执中输出 execution_mode='retained_decision_turn' 与 decision_capture_sha256;未传新参数时既有基准与主干行为保持不变。 本 PR 不改变普通主控制面会话压缩策略、不替代在线多轮 Core 写回闭环,也不在离线测试中发起真实远端模型推理。

改动思路 / Approach

本次实现只在 benchmark 能力层与独立 CLI 入口内部收敛,没有触碰通用 Core 控制面调度、Todo 或持久化回执状态。为了防止大体积捕获被原生工具输出上限静默截断,同时避免无效文件被送入模型上下文,作者在本地以 48_000 字节为硬阈值并与 thread/start 的 tool_output_token_limit: 48000 保持一一对应,再通过 thread/inject_items 的包裹结构写入 {"retention": "always", "item": {"type": "function_call_output", "name": "loopx_original_decision", ...}}。

该改动把一次性协议注入准确收口在现有 benchmark 工具包与原生评测入口之内,直接复用既有的上下文捕获校验,没有在主控制面制造平行状态或调度分支,职责边界清晰且易于维护或回退。 当前 PR 边界仅覆盖独立只读单轮基准评测入口中的原始决策保活注入(配合显式成对 CLI 参数),不包含主控制面会话恢复改动或在线远端推理实测。

对比维度 改动前 (origin/main) 改动后 (72510675d8)
入口激活 仅普通 preflight / native goal 路径 新增可选成对参数 --decision-capture 与 --decision-sha256(默认关闭)
输入校验 仅 NativeGoalConfig 与技能检查 额外强制绝对路径、decision.json 文件名、私有权限、SHA-256 与 <= 48_000 字节
线程配置 普通只读参数 ephemeral=True、read-only、关闭多 Agent、tool_output_token_limit=48000
注入语义 无压缩保留标记 thread/inject_items 发送 retention='always' 包裹 function_call_output

具体改动 / Changes

经核对 loopx/control_plane/ 与 PR 说明,本改动属于独立的 benchmark 可靠性增强,没有对应的预先编写的 RFC 或 Issue 规格(spec_basis=no_spec),故以仓库既有 decision_capture 与 native_codex_goal 契约为准绳进行逐项核验。全量变更共 4 个文件(+147/-1):运行时 loopx/capabilities/benchmark_toolkit/native_codex_goal.py(+64)与 benchmark/deepswe/run_native_codex_goal.py(+21/-1)、单测 benchmark/tests/test_native_codex_goal.py(+47)以及文档 benchmark/deepswe/README.md(+15)。

关键代码讲解

  1. _RETAINED_DECISION_BYTES(loopx/capabilities/benchmark_toolkit/native_codex_goal.py:40):将捕获字节上限固定为 48_000,同时约束注入前的体积检查与 thread/start 传入的 tool_output_token_limit,确保原始决策内容不会在工具输出注入阶段被静默裁剪。
  2. run_retained_decision_turn(loopx/capabilities/benchmark_toolkit/native_codex_goal.py:466-522):核心编排函数。先校验 config.validate()、拒绝与单轮定位无关的 token_budget/sandbox_policy/required_skill_ids、限制 0 < timeout_sec < inf 并校验 capture.is_absolute() 与文件名为 decision.json、检查 lstat().st_size <= 48_000 后,直接复用 decision_capture.read_context_capture_file(str(capture), digest, private=True) 完成私有目录/文件权限、防符号链接及 SHA-256 校验;随后完成 initialize(experimentalApi=True)、initialized、thread/start(校验服务端返回 ephemeral is True)、thread/inject_items(retention='always'),最后才进入 turn/start 与 wait_native_goal_turn。
  3. main 参数路由(benchmark/deepswe/run_native_codex_goal.py:58-59,95-98,156-163,193-196):要求 --decision-capture 与 --decision-sha256 必须成对出现,拒绝与 --preflight-only 或 --token-budget 混用;执行成功或失败均输出脱敏后的 native_codex_goal_turn_receipt_v0 回执并附加 execution_mode='retained_decision_turn' 与 decision_capture_sha256,并以 int(turn.turn_status != 'completed') 作为退出码。
  4. test_retained_decision_is_exact_tool_data_in_one_ephemeral_turn 与 test_retained_decision_refuses_invalid_original_before_native_input(benchmark/tests/test_native_codex_goal.py:35-78):覆盖正常协议序列、包裹载荷结构以及摘要篡改和 48_001 字节超限在发起任何 Codex RPC 前拦截的能力。

与仓库内同类 thread/inject_items 调用方(loopx/capabilities/native_chat/codex_adapter.py:998 用于长生命周期聊天会话恢复批量历史注入)对照,两者业务场景正交,本 PR 复用 read_context_capture_file、wait_native_goal_turn 与 StdioNativeGoalTransport,没有重复实现捕获解析逻辑。

对主干的风险 / Risks

  • Default-off 隔离性(isolated):新能力仅当显式同时传入 --decision-capture 与 --decision-sha256 时触发;两者默认均为 None,磁盘上存在捕获文件或升级 codex-cli 都不会被动激活新路径,未传新参数时原有调用链与回执契约保持不变。单一参数缺失或混用 --preflight-only/--token-budget 会在启动子进程前以退出码 2 快速失败。
  • 故障与边界回归面:所有前置条件与协议检查均按 fail-closed 原则执行——摘要不匹配、超 48_000 字节、非私有目录权限或服务端未返回 ephemeral=True 均在 turn/start 之前报错中止,也不会接触或修改任何 Goal/Todo/Registry 状态。
  • 比例性与 Scope Fit:共新增 +85/-1 行生产代码且与文档、单测严格同源;run_native_codex_goal.py 直接调用新函数,无死代码或冗余抽象。
  • 语义与权限契约(authority_semantics: aligned, semantic_alignment: not_applicable):execution_mode='retained_decision_turn' 等字段严格描述单轮只读临时会话行为,不夸大任何生命周期或持久化写回权限,也未修改主干共享状态枚举;examples/semantic-vocabulary-drift-smoke.py 已在 premerge 检查中通过。
  • 非阻塞观测记录:(1)benchmark/tests/test_native_codex_goal.py 未内置 run_native_codex_goal.py 命令行子进程单测(本次审阅已通过真实 CLI 反事实测试补齐验证);(2)未在本机离线复现真实远端大模型多轮推理触发的物理上下文压缩,协议结构本身已在本机真实 codex-cli 0.171.0-alpha.13 app-server 验证接受 retention='always' 并拒绝非法变体 'bogus'。

我的整体评价 / Verdict

综合判断,PR #9127 职责清晰、默认隔离干净、校验边界严密,无阻塞问题。

  • Outcome Impact:long_horizon=improved(在独立只读单轮基准评测中通过 retention='always' 与 tool_output_token_limit=48000 避免原始 LoopX 决策在压缩后丢失,同时不污染任何持久状态);user_experience=preserved(纯可选 CLI 入口与明确参数错误反馈,未破坏现有调用体验)。visual_review:Inspected benchmark/deepswe/run_native_codex_goal.py, loopx/capabilities/benchmark_toolkit/native_codex_goal.py, benchmark/deepswe/README.md, and benchmark/tests/test_native_codex_goal.py: CLI and Python runtime adapter change only with no rendered UI or visual presentation surface.;unverified_scope:none。
  • 代码体量与语义演化:共 4 个文件 +147/-1,其中新状态枚举/共享词汇 local_only(未新增跨模块共享词汇);兼容性无破坏(not_applicable)。
  • 验证汇总(精确 head 72510675d8a0437b6fa082d94f2de3ffeda9ba8c):
    • PASS:ruff check 全部通过;pytest benchmark/tests/test_native_codex_goal.py tests/control_plane/test_codex_context_capture.py 取得 22 passed, 46 skipped(22 项单测全过,46 项在线 Codex 场景按环境条件跳过);pytest benchmark/tests/test_native_codex_goal.py benchmark/tests/test_shared_codex_runtime.py 取得 139 passed, 72 skipped;loopx check --scan-path 通过(errors=0,scanned_files=5204);loopx canary premerge --from-git-diff --git-diff-base origin/main 全部 19 项运行检查通过(failures=0,仅因触及 benchmark_sensitive 面触发 1 项人工确认保留门禁);真实 CLI 子进程反事实(正常 48_000 字节回执、turn_status='failed' 退出码 1、参数冲突退出码 2)均符合预期;真实 codex-cli 0.171.0-alpha.13 app-server 验证 thread/inject_items 返回 {} 并严格拒绝 retention='bogus'(-32600)。
    • UNVERIFIED:未调用外部付费模型接口完成端到端真实推理中的长上下文自动压缩,这是离线只读审阅的最大剩余未覆盖验证项;但协议契约、本地摘要校验与入口隔离已充分证实。
  • 结论:无阻塞缺陷(no blocking finding),同意合入。

English verdict: APPROVE

@loopx-agent loopx-agent left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewer: model_agent · gpt-6.2-sol · OpenAI · runtime_reported · xhigh

动机 / Motivation

本次面向需要验证原决策在中途压缩后仍可见的 Native Codex 基准操作者。若普通工具输出在压缩后丢失,后续模型请求就可能缺少原 Turn 的约束和停止条件。我用实际 Codex App Server 和本地合成服务做了成对试验。两次都进入同一个 Turn 内的 contextCompaction,压缩前都能看到合成原文,压缩后只有显式保留的那次仍收到逐字节一致的内容。正式常驻 Host 的集成和使用效果留给独立验收。

改动思路 / Approach

原捕获以可核验的独立工具数据进入已有 Codex 传输,摘要和重新准入都不能替代这份原始输入。当前 PR 的边界是一次临时只读 Turn 的原捕获校验、注入和压缩后验真,沿用已有的事件等待与简洁回执。

依赖普通历史工具输出最省实现,但对照组已证明它可以随压缩消失。把决策重写为提示词或创建第二套续跑状态也会改变可检验的原文与身份。这里仅使用宿主现有的 retention 选项,输入需要显式路径和摘要,输出接回现有事件与 receipt。

具体改动 / Changes

审查提交为 7251067,基线 2000c39。四个文件净改动 +147/-1。main 必须同时收到 capture 和 SHA-256,和 preflight 或 native Goal budget 组合不成立。选择这个分支才启动单次 transport。run_retained_decision_turn 复用私有捕获读取和摘要检查,拒绝非原名、越界长度、错误摘要及非临时线程。在确认临时只读线程后,thread/inject_items 以 retained tool output 写入原字节,再发送一次 turn/start。后续复用 wait_native_goal_turn,receipt 返回摘要和终态。README 说明配对输入、支持宿主及压缩后的核对方式,测试保留了 JSON 非 ASCII、改动摘要、超长输入和旧模式的位置。实现路径见 runner 和 原文注入。

基线 required-work-context.md 对应三条本次约束。原始来源由捕获摘要和 UTF-8 内容绑定,缺失或变更必须拒绝,读回保持原准入权威边界。当前试验具体可验收项以 #9127 交付说明 为准。

对主干的风险 / Risks

无阻断发现。最大风险是测试只验注入请求,压缩以后却丢了原文。实跑 Codex 0.171.0-alpha.13,两个隔离会话各收到一次真实 contextCompaction 完成事件,均在同一 Turn。只有 retention=always 的会话在压缩后的第一个请求中保留了原 SHA-256 对应的完整内容。普通工具输出对照在相同位置找不到该原文。两侧各完成一个 Turn,未附加 native Goal。本地 provider 使用合成响应,模型目录提示使用回退元数据,这里验证原生窗口与工具数据传输。

本地 native runner 和 shared runtime 共 139 项通过、72 项按测试环境跳过。初次运行缺少测试夹具导入路径,修正调用环境后完整重跑。快速预合入本地检查通过,仍有 benchmark-sensitive 人工门禁。缺 SHA-256、内容被改动、超过输入上限等路径均有覆盖。本轮验证单次本地传输和实际压缩。生产模型效果、常驻宿主、其他平台,以及远端 CI 的结果需要各自证据。

我的整体评价 / Verdict

批准本次单轮实验增量。该方式在宿主真实发生窗口变化后仍能读回同一份 UTF-8 决策,减少测试依靠恢复提示或重放输入造成的混淆。这个功能明确选择,持久状态和普通 Goal 运行仍由原路径处理。人工合入门禁和常驻宿主范围保持分开。

本次涉及 benchmark CLI、只读单轮传输与说明,无渲染界面。未验证范围为实际远程模型、常驻 Host、Linux 与 Windows 的发布态及其效果和完整成本。

English verdict: APPROVE at 7251067. An actual Codex App Server with an offline fixture preserved the exact captured bytes on the first post-compaction request; the same plain tool item was lost.

@loopx-agent
loopx-agent merged commit e14e987 into main Nov 21, 2026
37 of 50 checks passed
@loopx-agent
loopx-agent deleted the codex/native-decision-pr-20261120 branch November 21, 2026 13:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants